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Abstract 

Renyi entropy of order a is a general measure of entropy. In this 
paper we derive estimations for the Renyi entropy of the mixture of 
sources in terms of the entropy of the single sources. These relations 
allow to compute the Renyi entropy dimension of arbitrary order of a 
mixture of measures. 

The key for obtaining these results is our new definition of the 
weighted Renyi entropy. It is shown that weighted entropy is equal to 
the classical Renyi entropy. 



1 Introduction. 

The investigation of data compression, coding and behaviour of statistical 
and physical systems involves the notion of the entropy [3, 5, 14, 18]. In 
information theory it is a basic term which is interpreted as a limit of best 
possible lossless compression of any communication. The reader interested in 
the history of information theory and data compression is referred to [1,17]. 

Renyi entropy of order a [12, 13] is a kind of measure of entropy which 
extends the notion of classical Shannon entropy [15]. The advantage of the 
Renyi entropy over the Shannon entropy lies in its generality. When modi- 
fying the parameter a it is possible to emphasise or weaken the relevance of 
some probability events [2]. Moreover, contrary to Shannon entropy, there 
exists efficient methods for computing Renyi entropy for some values of pa- 
rameter a [4]. 

The aim of this paper is to adapt the idea and results obtained recently for 
Shannon entropy [16] on the field of Renyi entropy of order a G (0, oo) \ {1}. 
The paper [16] provides the weighted approach to the Shannon entropy which 
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is based on measures instead of partitions. Weighted entropy describes the 
idea of random lossy coding which, from a practical point of view, can be 
more important than the classical deterministic coding.^ It is shown that 
this approach is equivalent to the classical one [16, Theorem II. 1]. It occurs 
that this alternative definition allows to derive some property concerning the 
Shannon entropy relatively easily. In particular, we obtain the estimation of 
Shannon entropy of the mixture of sources [16, Theorem III.l] and entropy 
dimension of convex combination of measures [16, Theorem IV. 1]. 

In our paper we show that the weighted entropy can be similarly defined 
for the Renyi entropy of any order a e (0, oo) \ {1} (precise definitions will 
be given in Section 2.2). As a main result we prove the equivalence between 
the definition of weighted and classical Renyi entropy (see Theorem 3.1). 
Consequently, we derive an optimal estimation of the Renyi entropy and 
Renyi entropy dimension of order a of the mixture of sources (see Theorem 
4.1 and 5.1). 

2 Basic definitions and their interpretations. 

In this paper, if not stated otherwise, we always assume that (X, E,//) is a 
probability space. 

2.1 Weighted Shannon entropy. 

To understand weighted approach to entropy, let us recall the basic concepts, 
definitions and facts from [16] where this idea is introduced for Shannon 
entropy. We use some of these definitions in the present paper. 

The problem of lossy data compression involves transformation between 
possibly uncountable set of data and some countable coding set. This coding 
mapping is usually obtained by defining a partition of a data set X. We say 
that a family P C E is a //-partition of X if P is countable family of disjoint 
sets and 

^Roughly speaking, random coding allows to code a given element once for one symbol 
and another time for other one. It is not necessary then to control precisely the way of 
coding. 
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Once the partition is chosen, the coding is defined precisely: we code a given 
X e X hy P e V iS X e P. Thus the partition is also called the coding 
alphabet. 

To obtain a statistical amount of memory per one element used in the 
lossy coding generated by partition V we use the Shannon /^-entropy of V 
defined by 

h{t^;V) J]/.(P)log2(//(P)). 

Pe-p 

The coding with use of a given partition causes specific level of error. To 
control the maximal error we make in the lossy coding we choose the error- 
control family Q which is simply the family of measurable subsets of X. We 
consider only such partitions V which are finner than Q i.e. we desire that 
for every P eV there exists Q E Q such that P G Q. li this is the case then 
we say that V is Q-acceptable and we write V ^ Q. 

Consequently, to describe the best lossy coding determined by Q-acceptable 
alphabets we define Shannon //-entropy of Q by: 

H{fi; Q) := inf P) E [0, cxd] : P is a //-partition and V -< Q}. 

Similar notions of the entropy in special cases were used by A. Renyi [12,13] 
and E. C. Posner [9-11]. They rather considered error-control families in 
metric spaces consisted of balls with given radius or cubes with specific edge 
length. 

The inspiration of weighted entropy, lies in the substitution of the division 
of space X into partition by the division of measure /x into "submeasures" ^ . 
It enables to use of the operations on functions rather than on plain sets. 
Roughly speaking, this approach provides the computation and interpreta- 
tion of the entropy with respect to "formal" convex combination aiPi -1-02^2, 
where Pi, P2 are partitions (which clearly does not make sense in the classical 
approach) . 

Let us denote the division of measure /i with respect to Q C S by: 

W{fx;Q):^ {m : Q 3 Q ^ mg E M{X,E) : 

rtiqiX \ Q) = for every Q e Q and Eqsq^^Q = A*}> 

where M(X, E) is the family of all measures on (X, E). Observe that every 
function m E W(/t; Q) is non-zero on at most countable sets of Q. Then we 

^Thc idea of weighted entropy is indebted to the notion of weighted Hausdorff measures 
considered by J. Howroyd [6,7]. 
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define the weighted /x-entropy of a given m G W{fj,; Q) by: 
/i^(/.;m) :=-5]mQ(X)log2(mg(X)). 

The sum in the above formula is taken over Q E Q such that mQ{X) > 0. 
The weighted entropy also can be generahsed for arbitrary family Q C S, 
taking the infimum over all functions of W{ijl; Q). 

It is shown [16, Theorem II. 1] that the weighted /x-entropy of Q is equal 
to the classical one i.e. Q). This equality allows to compute the entropy 

of the mixture of sources. Let us recall the motivation of this problem: 

Problem 2.1. Suppose that we are given two sources 5*1, 5*2, which are 
represented by probability measures /ii, respectively. We assume that the 
error-control family Q defines the precision in the lossy-coding elements of 
X. Let us consider a new source -S" which sends a signal produced by with 
probability Oi and produced by 5*2 with probability 02. We are interested in 
estimation of H{aifj,i + a2iJ>2', Q) (mixture of Si and ^'2) in terms of HifJ-i] Q) 
and H{i^2;Q). 

The following theorem gives the exact solution how to estimate the en- 
tropy of the mixture: 

Shannon entropy of the mixture [16, Theorem III.l]: Let Oi, 02 G [0, 1] be 
such that ai + a2 — I. If ^ii, 112 are probability measures and Q C E then: 

HiaiHi + a2Ai2; Q) > ai H{ij>i; Q) + a2 H{iJi2] Q) 

and 

H{aiixi + a2H2; Q) < ai H{ni; Q) + 02 if(/i2; Q) - ailog2(ai) - a2log2(a2). 



2.2 Weighted approach to the Renyi entropy of order 
a. 

In further parts of the paper we investigate the weighted approach to the 
Renyi entropy of order a. In this subsection we define precisely the weighted 
Renyi entropy. We use the idea from [16] described in the previous subsection. 
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Before that, we recall the classical definition of the Renyi entropy of order 
a for //-partition and its generalisation for family of measurable subsets of 
X. 

Definition 2.1. Let a G (0, oo) \ {1}. Given a //-partition P C E of X, 
Renyi jji-entropy of order aofV [13] is defined by 

For Q C E we define Renyi /^-entropy of order a of Q by 

Ha{f^; Q) ■— inf{/iQ,(/t; V) G [0, oo] : is a /t-partition and V -< Q}. 

Observe that if there is no /t-partition finer than Q then directly from the 
definition^ ^a(/^; Q) — oo. Moreover, if Q itself is a /t-partition of X then 
trivially H^ifJ,; Q) = /ia(/i; Q). 

As it was mentioned in the previous subsection, the partition describes 
the way of coding elements of X by the elements of some countable coding 
set. Given the maximal error we are allowed to make in the process of lossy 
coding, represented by the measurable family Q oi X, we consider all Q- 
acceptable partitions and choose the one which provides the lowest entropy. 

To see that Ha{fi; Q) does not have to be attained it is sufficient to use 
the trivial example from [16, Example II. 1]: 

Example 2.1. Let X = (0, 1), S be a sigma algebra generated by all Borel 
subsets of (0, 1), /i be a Lebesgue measure and Q be an error-control family 
defined by 

Q = {[a,b] : < a < & < 1}. 

Glearly i?a(A*; Q) — ^ but for every /i-partition V ^ Q,we have i?a(/(; Q) > 
when a G (0, oo) \ {1}. 

Inspired by the reasoning used in [16] we construct a definition of weighted 
Renyi /t-entropy of order a. The form of set W{fj,;Q) - a division of a 
measure /i with respect to error-control family Q, remains the same as in 
formula (1). The function form W{fi; Q) defines the set of measures. These 
measures determine the probability of encoding the given element with the 
code represented by the specific measure. Since every element x & X can be 
encoded once with one code and second time by another one, the coding is 
called random. 

^We put inf (0) = oo. 
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Definition 2.2. Let Q C S and let a E (0, oo) \{1}. We define the weighted 
Renyi ji-entropy of order a of m £ W{^] Q) by 

:= ^\og,{Y,mQ{Xr). (2) 

The weighted Renyi //-entropy of order o: of Q is defined by 

H'^{^Ji■ Q) := inf m) e [0, oo] : m e W{^i, Q)}. 

The sum in formula (2) is taken over Q E Q such that vn.Q{X) > 0, 
consequently this is a countable sum. 

The above definitions allow to perform the operations on functions when 
computing the entropy. Such operations will be crucial to derive some esti- 
mations of Renyi entropy and entropy dimension of the mixture of sources 
shown in the following sections. To use all the advantages of weighted Renyi 
entropy it remains to show the equivalence between weighted and classical 
entropy. It is made in the next section. 

We now make some additional notations and observations which we will 
refer to very often in future. We denote by ga and its inverse g~^ the following 
functions: 

g^{x) = 2(^-")^ g-\x) = J—log,{x). 

1 — a 

Then the observation is vahd: 
Observation 2.1. 1. If a e (0, 1) then: 

(a) g~^ and ga are ascending, 

(b) X ^ x"' is subadditive, 

(c) X — >■ is concave 

2. Ifae (l,oo) then: 

(a) g~^ and g^ are descending, 

(b) X ^ x^ is superadditive, 

(c) x — >■ is convex. 
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3 Equivalence between classical and weighted 
Renyi entropy of order a. 

The purpose of this section is to show that weighted Renyi entropy with 
respect to the family Q C S equals the classical Renyi entropy of Q. It will 
allow us to use benefits of this alternative definition in further analysis. In 
the proofs we apply the idea used in [16]. 

The equality will be derived in two steps. First we show the inequality 
H^{fi; Q) < H(y.{^\ Q). The inequality can be interpreted as a deterministic 
coding is a special case of specific random one. More difficult is to show the 
opposite inequality. It involves the application of Hardy-Littlewood-Polya 
Theorem. 

We start with first inequality: 

Proposition 3.1. Let a e (0, oo) \ {1}. Then 

H^il^; Q)<H^{fi; Q), 

for every family Q C E. 

Proof. Let us first observe that if there is no //-partition finer than Q then 
HaifJ'] Q) = oo and the inequality holds trivially. 

Thus let us assume that it is not the case. Let be a //-partition finer 
than Q. Our aim is to construct a function m e W{iJ,; Q) with lower entropy 
than V. 

First, since V -< Q, then we obtain a mapping n : V ^ Q such that 
P C n{P). Next we put 

T^Q {PQ}QeQ, 

where Pq := [j P. Finally, we define m : Q 3 Q ^ /X|Pq G M{X, S). 

P:n(P) = Q 

We verify that ra G W{iJ,; Q). Clearly, Vq is a //-partition and Pq G Q 
for every Q E Q. Hence 

QeQ QeQ QeQ 

The above sums are taken only over Q E Q such that mQ{X) > 0. Moreover, 
we have 

mgiX \ Q) = //|p^(X \ Q) < //|q(X \ g) = 0, 
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for Q e Q. We obtain that m G W{fj,; Q). 

It remains to check that /i^(/i;m) < /^^(/i;^). To see this we use Ob- 
servation 2.1: la and lb for a G (0, 1) or 2a and 2b for a e (l,oo). More 
precisely, we have: 



C(/";m) = ^log2( J]mQ(Xr) 

QeQ 

QeQ P:7r(P)=Q Q6QP:7r(P)=Q 

= iog2 ( E M^r) = ^a(/^; 7^). 

1 — a ^-^ 



Per 

As P was chosen as arbitrary partition, we conclude that H^{ii; Q) < 
Q) for a e (0,oo)\{l}. □ 

As it was mentioned to derive the inequahty [jj,; Q) > Ha{iJt.; Q) it is 
necessary to use Hardy Littlewood Polya Theorem. The version of Hardy 
Littlewood Polya Theorem for finite sequences is given in [8, Theorem 1.5.4]. 
Its generalisation for infinite sequences can be relatively easily achieved (see 
[16, Appendix A]). Let us recall this Theorem: 

Hardy Littlewood Polya Theorem. Let a > and let ip : [0, a] (0, oo), 

(y9(0) = &e a continuous function. Let {xi)i(=i, {yi)iei C [0, a] be given 
sequences where either I = N or I = {1, ... , N} for a certain N We 
assume that {xi)i^i is a nonincreasing sequence and 

n n 

Xi <'Y^yi ioi n e /, 

i=l i=l 

E^^ = E^- 

iei iei 

Then 

• Eie/ > Eie/ fiVj) if f is concave. 
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Eig/ Vi^i) < Y^iei viVj) if V is convex. 



We first show an additional proposition: 

Proposition 3.2. Let Q = {Qi}i£i be a family of measurable subsets of X, 
where either I — N or I — {1, . . . , N} for a certain N eN. Letm G W{iJ,; Q). 
We assume that 

. //(X \ U Q^) = 0, 

iei 

• the sequence / 9 i — >■ ttiq . (X) is nonincreasing. 

We define the family V = {Pi}ig/ C T. by the formula 

Pi := Qi, Pi := Qi \ U forieI,i> 2. 

k=l 

Then V is a ji-partition, V -< Q and 

C(/^;m)>/ia(//;P) (3) 

/orae(0,oo)\{l}. 

Proof. By the definition of family we get that V -< Q. Moreover, P is a 

//-partition since \ IJ Qi) — and \J Pi = [j Qi- 

iei iei iei 

To prove (3) we use Hardy Littlewood Polya Theorem. The sequences 

{xi)i(zi C [0, 1] and {yi)i(zi C [0, 1] are defined by the formulas 
Xi ■= mQ,{X) = mQ,{Qi), Vi := //(P^) 

for iei. 

Directly from the assumption we get that {xi)i^i is nonincreasing and 

^Xi = ii{X) = ^Vi- 
iei iei 

Moreover, for every n e P. 

n n n 

^ X, = ^ mg, (Q,) = (5^ J (Qi U . . . U Q„) 

i=l i=l 1=1 
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< MQi U . . . U Q„) = ^ = 5^ 

1=1 1=1 

We verified that sequences satisfy assumptions of Hardy Littlewood Polya 
Theorem. Thus given a function x — x° (we also use Observation 2.1: la 
and Ic for a e (0, 1) or 2a and 2c for a e (1, oo)) we conclude that 

e(/.;m) = ^log2 ( J]mQ,(X)-) = j^log^ij^x?) 

which completes the proof. □ 

We are now ready to formulate and complete the proof of the equivalence 
between classical and weighted Renyi //-entropy of order a. 

Theorem 3.1. Let Q C E. Then weighted Renyi /^-entropy coincides with 
the classical Renyi fj,-entropy, i.e. 

H^il,; Q) = Q) 

/orae(0,oo)\{l}. 

Proof. It is sufficient to show that H^{n; Q) > Ha{iJ,; Q) since the opposite 
inequality follows directly from Proposition 3.1. 

Let us first observe that if W{fj,; Q) — then H^{ii; Q) — oo and trivially 
H^ifi; Q) > H^ifi; Q). 

We discuss the case when W{n; Q) 7^ 0. Let m G W{ii; Q) be an arbitrary 
function. We define the subset of family Q by: 

Q:={QeQ: mgiX) > 0}. 

Let us notice that Q is a countable family since ^ mQ{X) = 1. Clearly, 

QeQ 

m := e W{n; Q). Moreover, Q ^ Q and /i^(/u;m) = /i^(/i;m). 

As Q is countable, we may find a set of indices / C N such that Q = 
{Qi}iei and the sequence I 3 i ^ m.Q.{X) is nonincreasing. Making use of 
Proposition 3.2 we construct a /i-partition V ^ Q, which satisfies 

h^{f,;m)>h^{f,;V). 

This completes the proof since V ^ Q ^ Q and h^{ii;m) = /i^(//;m) > 
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Since we show the cquahty between classical and weighted Renyi entropy 
we use one symbol H^^^fi; Q) to denote the Renyi entropy of order a with 
respect to measurable family Q of X. 

4 Renyi entropy of order a of the mixture of 
sources. 

In this section we will show how to apply the definition of weighted Renyi 
entropy to estimate the Renyi entropy of the mixture of sources (see Problem 
2.1). 

Let us start with the proposition: 
Proposition 4.1. We assume that a G (0, oo) \ {1} and n E N. Let au G 

n 

(0, 1) for k E {1, . . . ,n} be such that ^ = 1 and let {fJ^k}k=i Mi{X, S). 

k=l 

n 

We define jJi '■— Yli (^kl-i'k £ Mi{X, E). 

k=l 

• If V is a n-partition of X then V is a jik-partition of X for k e 
{1, . . . , n} and 

n 

haii^; V) >g-^[Y, (^k9{ha{i^k; V))] . (4) 

k=l 

n 

• If Q CT, and m'' e W(iJ,k; Q) for k e {1, . . . , n} then m := X) (^k^^ e 

fe=i 

W{fi; Q) and 

n 

h^{^^■xn)<g-'[J2at9{h^{^^k■,m'))]. (5) 
fe=i 

Proof. It is easy to see that V is a /x^-partition of X for every k E {1, . . . ,n}. 

Making use of Observation 2.1: la and Ic for a G (0, 1) or 2a and 2c for 
a e (1, oo), we have 

1 " 
^ Pev k=i 
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k=i Per 

1 " 

k=l 

n 

= [ E (^kgihaii^k; V))] , 

k=l 

which proves (4). 

We derive the second part of the Proposition. Clearly, m G Q). To 

verify (5) we use Observation 2.1: la and lb for a e (0, 1) or 2a and 2b for 
a e (1, oo): 

QeQ k=i 
k=i QeQ 

^ k=l 

n 

k=l 

□ 

Below we present the main theorem in this section concerning the entropy 
of the mixture of sources. To see that this estimation is sharp we refer the 
reader to Example 4.1. 

Theorem 4.1. Let a e (0, oo) \ {1} and n e N. We assume that e [0, 1] 
for k e {l,...,n} be such that Ofc = 1. Let {tik}k=i ^ Mi(X,E). // 

k=l 

Q C E then 

n 

Haifi; Q) > r ' [ E ^kdiHaifik, Q))] (6) 
k=l 
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and 

n 

H.il^; Q) <g-'[J2 <9(H^(f^k; Q))] ■ (7) 
fe=l 

Proof. Let us first consider the case when Ha{iJ>k', Q) — oo for a certain 
A; G {1, . . . , n}. Then also Ha{ii] Q) — oo and the inequahties hold trivially. 

Thus let us assume that for every k G {l,...,n}, Ha{fJ^k', Q) < oo. 
Without loss of generality, we may assume also that 7^ for every 

n 

k E {1, . . . , n}. Let e > be arbitrary and let jJ. '■= Yl C'kfJ'k- 

k=l 

To prove the first inequality, we find a //-partition V finer than Q such 
that 

H^ifi;Q)>h^ifi;V)-6. (8) 

Consequently, by Proposition 4.1 and the definition of Renyi entropy, we 
have 



fc=i 



k=l k=l 

Finally by (8), we obtain 

n 

Hail^] Q) > Kill; V)-e> g-'iJ^^'kgiHail^k] V))] - e, 

k=l 

which proves (6). 

We prove the inequality (7). For each k G {l,...n} we find m'^ G 
W{iik] Q) satisfying 

h'^{f,k;m>')<H^{fXk;Q) + -. (9) 
Making use of Proposition 4.1 and (9), we have 

n n 

Haifi; Q) <g-'[J2 <9ih^il^k; m'))] <g-'[J2 <9iHa{l^k; m^))] + e. 

k=l k=l 

This completes the proof as e > was an arbitrary number. □ 
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The following examples confirms that the above estimation (6) and (7) 
cannot be improved. 

Example 4.1. Let us assume that X — {0,1} and /Lti,//2 denote discrete 
measures such that: 



/.i({0}) = 1 and /X2({1}) = 1. 

Then, we have 

HaiaiHi + a2At2) = :r~~ log2« + 02)- 
1 — a 

It is exactly the right side of the inequality (7). 

On the other hand, if we consider two measures which satisfy /ii — 112, 
then 

Ha{ailli + a2fJ,2) = Ha{fJ,l) = Ha{ll2) 

and it equals the right side of (6). 

Let us observe the similarity between bounds obtained for both. Shannon 
entropy [16, Theorem III.l] and Renyi entropy of order a from Theorem 4.1. 
Let us consider the functions: 

Ua{x, y) = g~^{a'lga{x) + a^ga{y)), 

which describe the lower and upper bound for the Renyi entropy of order a. 

If X, y are non negative real numbers then these functions converge to the 
corresponding bounds calculated for Shannon entropy as a ^ 1 i.e.: 

r ^x. y)^arx + a2y when a ^ 1 

1 Ua{x, y) aix + a2y - ai log2(ai) - 02 log2(a2) ' 



5 Renyi entropy dimension of order a. 

In this section we show the estimation of Renyi entropy dimension of order 
a of the combination of measures in terms of the entropy dimension of the 

single measures. Before proceeding with it let us recall the definition of Renyi 
entropy dimension of order a. In this section we additionally assume that X 
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is a metric space and {X, E, fj,) is a probability space, where E contains all 
Borel subsets of X. 

Given J > let us denote a family of all balls in X with radius S by 

Bs := {B(x,5) -.xeX}, 

where B{x,6) is a closed ball centred at x with radius 5. 

Definition 5.1. The upper and lower Renyi entropy dimension of order 
a e (0, oo) \ {1} of measure /i e Mi{X, E) are defined by 



dim« (//):= lim sup 

5-^0 -log2(5) 

dim Jp) := liminf J^^i/fu^ 

If the above are equal we say that // has the Renyi entropy dimension of 
order a and denote it by dimQ,(//). 

The following theorem gives the estimation of the Renyi entropy dimen- 
sion of the mixture of measures. 

Theorem 5.1. Let ai,a2 G (0, 1) be such that ai + 02 = 1 and let //i,/i2 € 
Mi(X, E). If dimaini) < 00 and dima{iJ,2) < 00 then 



Ah^ (n n n \ < ) max{dim« (^i) , dim„ (/is) } for a e (0,1), 

[ mm{dimc,(//i),dima(//2)} /or a e(l, 00) 

anc? 

dimja,/., + a2/.2) > | ^^^i^i^^^)^ for a G (0, 1) 
av ip-i 2A*2y _ I mm{dim^(/ii),dim^(/i2)} /or a G (1, 00). ^ 

Proof. Let us show first inequality from formula (11). Rest of them can be 
proven in similar manner. 

Directly from the definition of Renyi entropy dimension of order a G 
(0, 00) \ {1}, we have: 

liminf -^"(^^'^^) ^ dim^(yUjfc), for k^l,2. 
<5^o -log2(d) 
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Then for arbitrary £i, £2 > 0, there exists 81,62 > 0, such that: 

-log2(dfc) 

and consequently 

H^dik, Bs^) > -log2(4)(dim^(/Xfe) 

for A; = 1,2. 

We put 5 := minj^i, 52}. Making use of Observation 2.1: la, we get: 

ga^iaiQa [ HaifJ^i; B5)] + a2^a [ -f^a(A*2; Bs)] } 

> 9a^{ai9a[- log2(5)(dim^(/ii) - £i)] + a2^a[ - log2(5)(dim^(//2) - £2)] }• 
By Theorem 4.1, we have: 

Ha{aiiJ.i + a2iJ,2;Bs) > g~^{aiga[Ha{iJ,i;Bs)] + a2ga[Ha{iJ>2;B5)]} 

> 9a^{aiga[-'^og2i5){^m^{ni) -£i)] +a2^a[-log2((^)(dim^(//2) -£2)]}- 

Dividing the above inequahty by (— log2(5)) and taking the hmit as 5 — >■ 0, 
we conclude: 

. r Ha{aifii + a2H2;Bs) 
lim mt 



> liminf 



5^0 -logs (5) 

g-^{aiga[-log2{S){^^{Hi) - El)] + a2^„ [ - log2(5)(dini^(/X2) - £2)] } 



5^0 -log2(5) 



lim inf 



= lim inf 



5^0 -l0g2((^) 

"Ao" -log2((^) 

= dim„(yui) - £1. 

Since £1,62 was the arbitrary numbers, then desired inequality holds. □ 

Clearly, the above theorem can be generahsed for any finite number of 
measures. In the case when all measures have Renyi entropy dimension of 
order a then the entropy dimension of the convex combination of measures 
is determined precisely. 
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n 



Corollary 5.1. Let G (0, 1) for k = 1, . . . ,n he such that X] <^fe = 1 

k=l 

where n e N and let {nkyk=i C Mi(X, S). // every /ik has finite Renyi 
entropy dimension for k & {1, . . . ,n} then Ylk=i l^k o-lso have Renyi entropy 
dimension. Moreover, 
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